Dynamic Management of Caching Tiers

Application owners are typically concerned about getting the best performance at the lowest cost. When the workload demand for an application is dynamic, lowering the cost necessitates dynamic management of the hosting infrastructure (physical servers or virtual machines). An application deployment is often divided into multiple tiers. The frontend tier is usually stateless, and processes incoming workload requests using data from the backend. The backend tier is stateful, and typically consists of a persistent storage system, such as a database. To alleviate load at the backend, a stateful, but non-persistent, caching tier is often used to cache data or partial results. For such multi-tier deployments, the ultimate goal is to scale all of the tiers to match the varying workload demand. Stateless tiers are easy to scale (see, for example, [7, 6]) since they do not store data that is required for request processing. Scaling the stateful data tier is more complicated. For the database tier, there needs to be at least one copy of the data, so research is limited to questions of how many replicas are needed and where to place the data (see, for example, [2, 13]). The caching tier, on the other hand, stores only a fraction of the total data, and this data is not necessarily replicated. Since the caching tier is non-persistent, we are not limited by a minimum amount of cached data, so there is significant potential for scaling down the caching tier. However, the caching tier plays a crucial role in filtering the amount of requests that go to the database tier. Thus, dynamically managing the caching tier requires careful consideration of the possible impact on end-to-end application performance. To the best of our knowledge, there has been no work on dynamically managing the caching tier. As part of this thesis, we propose two orthogonal approaches to dynamically managing the caching tier: (i) CacheScale [14], and (ii) SOFTScale [8]. CacheScale works by dynamically scaling the caching tier in response to changes in workload demand. To avoid performance degradation due to transient cache misses, CacheScale carefully redistributes important data elements prior to scaling. SOFTScale, on the other hand, does not scale the caching tier, and instead leverages the spare capacity in the caching tier nodes to handle some of the stateless tier load. SOFTScale is especially useful for handling transient overload caused by a sudden increase in demand.